Electronic apparatus having a sensing unit to input a user command and a method thereof

ABSTRACT

A mobile device includes a housing, an input and output unit including a panel having a display area on a first side of the housing and configured to display an image or menu to a user and to receive a user input from the user to perform a function of the mobile device, a sensing unit disposed between the panel and a second side of the housing to sense a first object, which is disposed to face the display area of the panel, through the panel, and a controller disposed in the housing and configured to perform a sensing mode which includes a sensing operation of the sensing unit to sense the first object through the panel and to generate a sensed image from the first object, and to determine a state of the sensed image as another user input to perform another function of the mobile device.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Continuation Application, under 35 U.S.C. §120, ofU.S. Non-Provisional application Ser. No. 15/875,367, filed on Jan. 19,2018, which is a Continuation Application of a Non-Provisionalapplication Ser. No. 15/440,033, filed on Feb. 23, 2017, now U.S. Pat.No. 10,013,115, which is a Continuation Application of a Non-Provisionalapplication Ser. No. 15/246,302, filed on Aug. 24, 2016, now U.S. Pat.No. 9,864,511, which is a Non-Provisional application Ser. No.14/583,537, filed on Dec. 26, 2014, now U.S. Pat. No. 9,454,235, in theUnited States Patent and Trademark Office, the contents of which areincorporated herein in their entirety by reference.

BACKGROUND OF THE INVENTIVE CONCEPT 1. Field of the Inventive Concept

The present inventive concept relates to an electronic apparatus andmethod of inputting a user command through a sensing unit thereof.

2. Description of the Related Art

A conventional electronic mobile device, for example, a cellular phone,includes a camera and a display unit having a display area to display animage photographed by the camera. However, a center line of the cameradoes not correspond to a center line of the display area. Therefore,when an object is disposed over a position of the center line of thedisplay area, an image of the object is displayed on a positiondifferent from the center line of the display area of the display unit.

When a user intends to place an object on a position to be disposed overthe display area to correspond to a position of the display area in avertical direction relationship between the position of the object andthe position of the display area, an image of the object displayed onthe display area is deviated from the intended position.

When a user intends to move an object between positions to be disposedover the display area to correspond to positions of the display area ina vertical direction relationship between the positions of the objectand the positions of the display area, an image of the object displayedon the display area is deviated from the intended positions.

Since a position of an object is sensed to be different from a positionof a display area, it is difficult to sense a position of an objectsensed by a camera to correspond to a position of a point of a displayarea.

SUMMARY OF THE INVENTIVE CONCEPT

The present inventive concept provides an electronic apparatus to inputa user command through a sensing unit having a virtual area overlappinga display area in a vertical direction of the display area in anelectronic apparatus.

The present inventive concept provides a method of inputting a usercommand through a sensing unit having a virtual area overlapping adisplay area in a vertical direction of the display area in anelectronic apparatus.

The present inventive concept provides a computer readable medium tocontain computer readable codes as a program to execute a method ofinputting a user command through a sensing unit having a virtual areaoverlapping a display area in a vertical direction of the display areain an electronic apparatus.

Additional features and utilities of the present inventive concept willbe set forth in part in the description, which follows and, in part,will be obvious from the description, or may be learned by practice ofthe present inventive concept.

The foregoing and/or other features and utilities of the presentinventive concept may be achieved by providing a mobile device includinga housing having a first side and a second side, an input and outputunit comprising a panel having a display area on the first side of thehousing, wherein the display area of the panel is configured to displayan image or menu to a user and to receive a user input from the user toperform a function of the mobile device, a sensing unit disposed betweenthe panel and the second side of the housing to sense a first object,which is disposed to face the display area of the panel, through thepanel, and a controller disposed in the housing and configured toperform a sensing mode which includes a sensing operation of the sensingunit to sense the first object through the panel and to generate asensed image from the first object, and to determine a state of thesensed image as another user input to perform another function of themobile device.

The mobile device may further include a photographing unit disposed onthe housing and comprising one or more cameras to photograph a secondobject, and the controller may be configured to perform a photographingmode which includes a photographing operation of the photographing unitto photograph the second object and controlling the panel to display thephotographing second object.

The panel may include a touch panel to display the photographing secondobject and to receive a touch input as the user input on the displayarea, and the sensing unit may include a sensing element disposed on aside of the panel opposite to the display area of the panel such thatthe sensing unit senses the first object through the panel and generatesthe sensed image as the another user input.

The mobile device may further include a photographing unit disposed onthe housing and comprising one or more cameras to photograph a secondobject and to generate a photographing image in a photographing mode,the photographing image may be displayed on the display area of thepanel, and the sensed image may not be displayed on the display area ofthe panel.

The controller may control the panel to display a symbol correspondingto the sensed image on the display area of the panel.

The panel may display another menu, the sensing unit may sense the firstobject, which is disposed to correspond to the another menu, through thepanel to generate the sensed image, and the controller may perform theanother function corresponding to the another menu according to thestate of the sensed image.

The sensing unit may include a sensing element disposed at a bottom ofthe panel opposite to the display area to sense the first object throughthe panel.

The sensing unit may have a first center line parallel to a secondcenter line of the panel.

The panel may include a liquid crystal device including a light elementand a circuit element to form the image or menu on the display area, andthe sensing unit may include a sensing element disposed between thecircuit element of the liquid crystal device and the second side of thehousing.

The panel may include a liquid crystal device including a circuitelement and a light emitting element to generate light toward thecircuit element to form the image or menu, and the sensing unit mayinclude a sensing element disposed at a position corresponding to thecircuit element of the liquid crystal device to sense the first objectthrough the panel.

The panel may include a light emitting device unit including a firstportion with a first number of light emitting elements and a secondportion with a second number of light emitting elements, to form theimage or menu, and the sensing unit may include a sensing elementdisposed at a location corresponding to the first portion of the panel.

The sensing unit may include a sensing element, and the panel mayinclude a portion corresponding to a location of the sensing element ofthe sensing unit, and is configured to provide a condition to theportion when the sensing element of the sensing unit senses the firstobject through the panel in the sensing mode.

The panel is configured to prevent an interference with the sensingoperation of the sensing unit when the sensing unit senses the firstobject through the panel in the sensing mode.

The sensing unit may include a sensor to sense the first object throughthe panel, the panel may include a first portion and a second portioncorresponding to a location of the sensing element of the sensing unit,and the first portion and the second may be different in transparencycondition.

The first object may include a plurality of sub-objects, one of whichhas a higher priority than the other one. The controller may beconfigured to generate another sensed image corresponding to the onesub-object with the higher priority, and may be configured to determineat least one of the state of the sensed image and a state of the anothersensed image of the one sub-object with the higher priority as theanother user input to perform the another function of the mobile device.

The first object may include a plurality of sub-objects, one of whichhas a higher priority than the other one, the sensing unit may sense theplurality of sub-objects through the panel and generates another sensedimages corresponding to the other sub-object and the one sub-object withthe higher priority, and the controller may be configured to determineat least one of the state of the sensed image and states of the anothersensed images as the another user input to perform the another function.

The mobile device may further include an interface disposed in thehousing and configured to communicate with an external device totransmit or receive data, and the controller may be configured toprocess the data according to the state of the sensed image and tocontrol the interface to transmit the processed data to the externaldevice.

The controller may be configured to extract a portion of the sensedimage and determine a state of the extracted portion as the another userinput.

The controller may be configured to modify the sensed image to generatea state of the modified image as the another user input.

The sensing unit may include one or more sensing elements disposed atcorresponding sensing areas, at least one of the one or more sensingelements may sense the first object through the panel, and thecontroller may generate the sensed image and determines the sensed imageas the another user input.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other features and utilities of the present generalinventive concept will become apparent and more readily appreciated fromthe following description of the exemplary embodiments, taken inconjunction with the accompanying drawings of which:

FIG. 1 is a diagram illustrating an electronic apparatus according to anembodiment of the present inventive concept;

FIG. 2 is a block diagram illustrating an electronic apparatus accordingto an embodiment of the present inventive concept

FIGS. 3 and 4 are views illustrating a sensing unit and a display screenin an electronic apparatus according to an embodiment of the presentgeneral inventive concept;

FIGS. 5 and 6 are views illustrating an object sensed by a sensing unitof an electronic apparatus according to an embodiment of the presentinventive concept;

FIGS. 7 and 8 are views illustrating a point of a virtual area tocorrespond to a display point of a display area in an electronicapparatus according to an embodiment of the present inventive concept;

FIGS. 9 and 10 are views illustrating a status of an objectcorresponding to a virtual point with respect to a virtual area and adisplay area in an electronic apparatus according to an embodiment ofthe present inventive concept;

FIGS. 11 and 12 are views illustrating objects sensed by a sensing unitin an electronic apparatus according to an embodiment of the presentinventive concept;

FIG. 13 is a view illustrating a location of a sensing unit in anelectronic apparatus according to an embodiment of the present inventiveconcept;

FIGS. 14A, 14B, 15A, and 15B are views illustrating a virtual area of asensing unit and a display area of a display unit according to anembodiment of the present inventive concept;

FIG. 16 is a view illustrating a location of a sensing unit in anelectronic apparatus according to an embodiment of the present inventiveconcept;

FIG. 17 is a view illustrating a virtual area of a sensing unit and adisplay area of a display unit according to an embodiment of the presentinventive concept;

FIGS. 18A and 18B are views illustrating a virtual point displayed on adisplay area of a display unit in an electronic apparatus according toan embodiment of the present inventive concept;

FIGS. 19 and 20 are flowcharts illustrating a method of inputting a usercommand using a sensing unit in an electronic apparatus according to anembodiment of the present inventive concept;

FIG. 21 is a view illustrating a location of a sensing unit in anelectronic apparatus according to an embodiment of the present inventiveconcept; and

FIGS. 22 and 23 are views illustrating an electronic apparatus having asensing unit installed thereon according to an embodiment of the presentgeneral inventive concept.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Reference will now be made in detail to exemplary embodiments of thepresent general inventive concept, examples of which are illustrated inthe accompanying drawings, wherein like reference numerals refer to thelike elements throughout. The exemplary embodiments are described belowin order to explain the present general inventive concept whilereferring to the figures.

FIG. 1 is a diagram illustrating an electronic apparatus 100 accordingto an embodiment of the present inventive concept. The electronicapparatus 100 may be a mobile device having one or more camera units tophotograph an object disposed in a front side and/or a back side of themobile device. The electronic apparatus 100 includes a housing 101, adisplay unit 102 disposed on the housing 101 to display an image on adisplay area DA having a display center line DCL, and a sensing unit 103having a sensing element to sense an object disposed around theelectronic apparatus 100.

The sensing unit 103 may include the one or more camera units having asensing center line SCL to generate an image corresponding to thephotographed object with respect to the display area DA. The sensingcenter line SCL and the display center line DCL are spaced apart fromeach other. Although the sensing center line SCL is illustrated in aleft-right direction of FIG. 1, the sensing center line SCL may bedisposed at a line perpendicular to a surface of the housing 101, forexample, the display area DA. Although the display center line DCL isillustrated in a left-right direction of FIG. 1, the display center lineDCL may be disposed at a line perpendicular to a surface of the housing101, for example, the display area DA.

As an example, when the sensing unit is a camera to photograph anobject, and the display area DA displays an image corresponding to thephotographed object, a center of the displayed image of the photographedobject may be not identical to a center of the photographed object dueto a distance between the sensing center line SAC and the display centerline DCL. When an object approaches toward a portion of the display areaDA in a direction perpendicular to a surface of the display area DA, theimage of the photographed object is not displayed to move toward theportion of the display area DA in a direction perpendicular to a surfaceof the display area DA. As another example, when an object moves towardor disposed at a center of the display area DA, the image of thephotographed object is not displayed at the center of the display areaDA but is displayed at another portion spaced apart from the center ofthe display area DA.

When a user is disposed in front of the electronic apparatus to face thedisplay area DA and a user finger or tool is disposed over or movestoward a point of the display area DA, a photographed image of the userfinger or tool is displayed on the display area DA differently as if theuser finger or tool is disposed over or moves toward another point ofthe display area DA. The photographed position (location) or movement ofthe user finger or tool may be referred to a virtual position (location)or movement of the object, and the position (location) or movement ofthe object with respect to the display area DA may be referred to anactual or physical position (location) or movement of the object. Thevirtual position or movement of the object in the photographed or sensedimage with respect to the display area DA may be different from theactual position or movement with respect to the display area DA.Therefore, the electronic apparatus may not determine or recognize thestatus of the object as a user input according to the virtual positionor movement in the photographed or sensed image but may generate anerror in determining or recognizing the virtual position of movement ofthe object as a user input with respect to the display area DA.Accordingly, the virtual position (location) or movement of thephotographed image of the object may be modified or corrected to theactual or physical position (location) or movement such that the statusof the object can be determined as a user input with respect to a pointof the display area DA. This modification or correction allows theelectronic apparatus to determine or recognize a user input from thephysical location or movement of the object with respect to the displayarea DA.

FIG. 2 is a block diagram illustrating an electronic apparatus 200according to an embodiment of the present inventive concept. Theelectronic apparatus 200 may be the electronic apparatus 100.

The electronic apparatus 200 includes a controller 210, a memory unit220, an interface 230, a sensing unit 203, and an input/output unit 204.The controller 210, the memory 220, the interface 230, the sensing unit203 and the input/output unit 204 may be included in a single housing.However, it is possible that at least one of the memory 220, theinterface 230, the sensing unit 203 and the input/output unit 204 may beconnected to the housing. The sensing unit 203 may correspond to thesensing unit 103 of FIG. 1, and the input/output unit 204 may correspondto the display unit 102 of FIG. 1. The memory unit 220 may store datausable to perform functions of the electronic apparatus 200. The memoryunit 220 may be a semiconductor memory device, a read-only memory (ROM),a random-access memory (RAM), a flash memory, a semiconductor chippackage, CD-ROMs, magnetic tapes, floppy disks, and optical data storagedevices. The input/output unit 204 may be a display device to display animage or a touch panel to receive a user input. The input/output unit204 may include an audio element to generate an audio signal alone ortogether with an image displayed on a display unit according to a userinput or data processed in the controller 210.

The sensing unit 203 may include one or more cameras to photograph anobject and/or a sensing element to sense an object disposed over adisplay area of the display device or touch panel. The photographed orsensed object may be displayed as an image on the display area DA.However, it is possible that the photographed or sensed object may notbe displayed as an image but may be usable as a user input (command ordata) to correspond to a portion (icon or menu) of the display area DA.It is also possible that the photographed or sensed object may not bedisplayed as an image of the photographed object but may be displayed asa symbol representing the photographed or sensed object with respect tothe display area DA. That is, when the sensing unit 203 is in a cameramode, the photographed object is displayed as an image on the displayarea DA. When the sensing unit 203 is in a sensing mode, the sensedobject is usable as a user input (or user data input) to correspond to aportion (icon or menu selection or a user command) of the display areaDA. It is also possible that in a combination mode, the photographedobject is displayed on the display area DA, the icon or menu isdisplayed on the display area DA, and the photographed (or sensed)object is used as a user input to control or perform a function of theicon (or menu) of the display area or to control the displayed image orto control the sensing unit 203. The camera mode, the sensing mode, andthe combination mode can be selected in a menu screen of the displayarea DA or can be selected according to a selection of a buttoncorresponding to the respective mode according to a user input.

The input/output unit 204 may be a display unit to display an image,and/or an input unit to input a user input (command or data). Theinput/output unit 204 may have a display area DA of a screen (panel) todisplay an image photographed by a camera, to display a menu icon, toreceive a user touch as a user input through the displayed menu icon,and to display an image sensed by the sensing element. The panel may bea touch panel. The touch panel is well known, detail descriptionsthereof are omitted. The input/output unit 204 may separately orindependently generate or receive the photographed image, the displayedmenu icon, the received user input, and the sensed image incorresponding modes according to a user design or preference. It ispossible that the input/output unit 204 may simultaneously generate orreceive the photographed image, the displayed menu icon, the receiveduser input, and the sensed image in corresponding modes according to auser design or preference. It is also possible that the input/outputunit 204 may generate a combination of the photographed image, thedisplayed menu icon, the received user input, and the sensed imageaccording to a user design or preference.

The controller 210 may generate a menu option to be displayed on thedisplay area DA. The menu option may include a first menu option toselect a camera mode in which a camera photographs an object andgenerates the photographed image to be displayed on the display area DA,a sensing mode in which a sensing unit senses an object disposed overthe display area DA and generates the sensed image corresponding to thesensed object to be displayed on the display area DA, and a combinationmode in which a camera is usable as a camera and a sensing element toperform both functions of the camera and the sensing element. Here, thephotographed image is an image photographed by a camera and the sensedimage is a symbol of the sensed object. Therefore, when one or moreobjects are photographed, the photographed objects are displayed on thedisplay area DA, and at least one of the photographed objects may besensed as the sensed image to be displayed as a symbol representing atleast a portion of the one object. The photographed image and the sensedimage may overlap on the display area DA. And also the menu icon and thesensed image may overlap on the display area DA. The sensed image may beuseable as a user input corresponding to a menu icon displayed on thedisplay area DA of the screen so that the controller 210 can determinethat a menu icon is selected according to a location of the sensed imageand can perform a function of the menu icon corresponding to the sensedimage.

The controller 210 may receive the photographed (or sensed) object fromthe sensing unit 203 and generate an image corresponding to thephotographed object to be displayed on the display area. Here, it ispossible that the controller 210 controls or modifies the image todisplay a portion of the photographed object which is disposed above thedisplay area DA in a direction, for example, in a directionperpendicular to the display area DA such that a center line of thedisplayed portion of the photographed object is identical to the centerDCL of the display area DA or such that an area of the display portionof the photographed object is identical to the display area DA. Thesensing unit 203 may perform at least a portion of the above describedfunction of the controller 210. According to status of the photographedor sensed object, the controller may receive a user input or command toperform a function including, but not limited to, a drag operation, aselecting operation, a data input operation, a displayed imagemodification operation, an communication method selecting operation, adata transmission or reception method selecting method, an externaldevice selecting operation, an access point selecting operation, and soon. The status of the object may be a stationary or a motion between twopositions in a virtual area VA of the sensing unit 203 with respect tothe display area DA of the input/output unit 204.

As stated above, due to different center lines of the display area DAand the sensing unit 203, a location or movement of the photographedobject with respect to the display area DA is displayed at a differentlocation or movement within the display area DA. When an object isdisposed at a position over a center of the display area DA in adirection perpendicular to the display area DA, the photographed objectis displayed at a different position from the center of the display areaDA. However, the photographed object disposed at a position over acenter of the display area DA in the perpendicular direction may bedisplayed at the center of the display area DA by analyzing andmodifying a photographed object image. The controller 210 may control,adjust, or modify a location or movement of an object photographed withrespect to the sensing unit 203 to correspond to a location and amovement with respect to display area DA. In this case, an objectdisposed at another portion which is not in an area corresponding to thedisplay area DA in the perpendicular direction may not be displayed inthe display area. An object disposed in an area corresponding to thedisplay area DA in the perpendicular direction may be displayed in thedisplay area. The sensing unit 203 may perform at least a portion of theabove described function of the controller 210.

The controller 210 may include a look-up table usable to analyze a firststate of the object in the photographed or sensed object in the virtualarea VA and to convert (correct or modify) the first state of the objectinto a second state of the object with respect to the display screen DAas the status of the object. The look-up table may include informationon coordinates of the virtual area VA and the display screen andinformation on a virtual location (or movement) of the object in thephotographed or sensed image in the virtual area VA and an actual orphysical location (or movement) of the object with respect to thedisplay area DA. When the object is disposed or moved in the virtualarea VA, the sensing unit 203 photographs the object to generate thephotographed or sensed image of the object with respect to thecoordinate of the virtual area VA. And then a location or movement ofthe photographed or sensed image of the object is analyzed and modifiedto a different location or movement with respect to the coordinate ofthe display screen. Since the coordinates of the virtual area VA and thedisplay area DA are same, the object is represented in the coordinate ofthe display area DA at a different location or movement from thelocation or movement in the virtual area VA. Although the coordinates ofthe virtual area VA and the display area DA are same, the virtuallocation or movement of the object photographed or sensed with respectto the virtual area VA may be different from the actual or physicallocation or movement of the object with respect to the display area DAaccording to a distance between the sensing center line SCL of thesensing unit 203 and the display center line SCL of the input/outputunit 204 or according to a distance from the sensing center line SCL tothe location of the object in the virtual area VA.

The interface 230 communicates with an external device 260 using a wiredor wireless communication method to transmit data to the external device260 and to receive data from the external device 260 according to a userinput. The controller 210 controls the above described units andelements of the electronic apparatus 200. The controller 210 may includeone or more semiconductor processors to perform functions of theelectronic apparatus 200.

FIG. 3 is a view illustrating a sensing area SA and a display area DA inan electronic apparatus according to an embodiment of the presentgeneral inventive concept. A camera (or sensing element) C1 as thesensing unit 203 of FIG. 2 photographs an object in the sensing area SAhaving a photographing angle “a” and a sensing center line SCL. Avirtual area VA is disposed in the sensing area SA to correspond to thedisplay area DA in a direction perpendicular to the display area DA. Thesensing center line SCL of the camera C1 may be parallel to a displaycenter line DCL of the display area DA. It is possible that sensingcenter line SCL of the camera C1 may not be parallel to a display centerline DCL of the display area DA to control or adjust a distance d1 whichis a distance between the display area DA and a boundary of a virtualplane VP or a boundary SAb of the sensing area SA. The distance D1 maybe adjustable when the camera C1 is adjustably installed in the housing101 of FIG. 1 to change the sensing center line SCL.

FIG. 4 is a view illustrating a sensing unit and a display screen in anelectronic apparatus according to an embodiment of the present generalinventive concept. The sensing unit 203 may have the camera C1 of FIG. 3to have a first sensing center line SCL1 and a first sensing area SA1,and a camera C2 to have a second sensing center line C2 and a secondsensing area SA2. The virtual plane VP may be disposed in an overlaparea of the first sensing area SA1 and the second sensing area SA2. Thevirtual plane VP may be disposed between the first sensing center lineSCL1 and the second sensing center line SCL2. The virtual area VA isdisposed over the display area DA in a perpendicular direction withinthe overlap area. the virtual plane VP may be spaced apart from thedisplay area DA by a distance d2. The distance d2 may also be adjustableaccording to an adjustably installation of the camera C1 and the cameraC2.

FIGS. 5 and 6 are views illustrating an object sensed by a sensing unitof an electronic apparatus according to an embodiment of the presentinventive concept. One or more icons (or menus) Ma, Mb, . . . , Mn maybe displayed on the display area DA. the virtual area VA is disposedover the display area in a perpendicular direction. When an object, forexample, a user's finger Pob is disposed over the virtual area VA ormoves toward or away from the virtual area VA. The sensing unit 203senses the object disposed in the virtual area VA, and the controller210 may determine whether a location (or position) of the object isdisposed to correspond to one of the icons Ma, Mb, . . . , Mn of thedisplay area DA.

When the controller 210 determines that the object is disposed at alocation (or position) Vpa corresponding to the icon Ma of the displayarea DA, the controller 210 may determine that the icon Ma is selectedso that a corresponding function of the icon Ma can be performed. Thecontroller 210 may determine the presence or existence of the object inthe virtual area VA as a user input. The object can be represented by astationary object disposed at a position or a movable object movablebetween two positions. The controller 210 may control the input/outputunit 204 to display a character or sign Tpa within the correspondingicon Ma of the display area DA to represent that the icon Ma isselected. It is possible that the icon Ma is highlighted or emphasized.The controller 210 processes the function corresponding to the userinput and controls the input/output unit 204 to generate an output, forexample, displaying an image corresponding to the processed function,communicating with other device, or transmitting or receiving dataaccording to the processed function.

Since the display area DA includes a touch screen, the controller 210may receive a user input from a touch input though the input/output unit204 and/or a sensed object though the sensing unit 203. For example, atouch input on the icon MA of the display area DA and a sensed object inthe virtual area are determined as a user input to control theelectronic apparatus.

FIGS. 7 and 8 are views illustrating a point of a virtual area tocorrespond to a display point of a display area in an electronicapparatus according to an embodiment of the present inventive concept.An object, for example, a user finger Pob, moves toward or disposed at aposition VPa or VPb with respect to a display center line DCL of thedisplay area DA in the virtual area VA. The positions VPa and TPa ofFIG. 7 are opposite to the positions VPb and TPb of FIG. 8 with respectto the display center line DCL of the display area DA. All of thepositions VPa, VPb, TPa, TPb are disposed in a side of the sensingcenter line SCL of the sensing unit 203.

FIGS. 9 and 10 are views illustrating a status of an objectcorresponding to a virtual point with respect to a virtual area and adisplay area in an electronic apparatus according to an embodiment ofthe present inventive concept. When an object moves between twopositions with respect to a position VPa in a first direction of FIG. 9or in a second direction of FIG. 10, the controller 210 determines thepresence, existence, or movement of the object as a user input to selectthe icon Ma, and then perform the corresponding function. Although FIGS.9 and 10 illustrate two directions, the present general inventiveconcept is not limited thereto. The controller 210 may detect themovement of the object in a direction other than the two directions ofFIGS. 9 and 10.

FIGS. 11 and 12 are views illustrating objects sensed by a sensing unitin an electronic apparatus according to an embodiment of the presentinventive concept. When the sensing unit 203 senses two or more objects,the controller 210 may provide a priority to at least one of objects todetermine a user input. For example, when there are two objects in thevirtual area VA, the controller 210 may determine that a user finger Pobhas a higher priority than a user face as a user input. It is alsopossible that a distal end T of the user finger Pob can have a higherpriority than other portions of the user finger Pob as a user input. Theuser input can be determined according to the presence, existence, andmovement of the distal end T in the virtual area.

FIG. 13 is a view illustrating a location of a sensor of a sensing unitin an electronic apparatus according to an embodiment of the presentinventive concept. The sensing unit may be a camera having a lens unit,and the lens unit may have a photographing angle to define a sensingarea SA and a sensing center line SCL disposed at a center of thesensing area SA. The sensing area SA may have two sensing areas SAa andSAb which are disposed opposite to each other with respect to thesensing center area SCL. The lens unit may have a sensor which may notdisposed at a center area but disposed at a side Sa or Sb with respectto the sensing center line SCL to sense one of the two sensing areas SAaand SAb. In this case, one of the two sensing areas SAa and SAb may notbe sensed by the sensor disposed at the side.

FIGS. 14A, 14B, 15A, and 15B are views illustrating a virtual area of asensing unit and a display area of a display unit according to anembodiment of the present inventive concept. The virtual area VA todetect an object as a user input is disposed at one side of the sensingarea SA, for example, the virtual plane VP, with respect to the sensingcenter line SCL. The sensing area SA and the display area DA areillustrated in a vertical direction in FIGS. 14A and 14B and in ahorizontal direction in FIGS. 15A and 15B, according to a location ofthe sensing unit with respect to the display area DA.

FIG. 16 is a view illustrating a location of a sensing unit in anelectronic apparatus according to an embodiment of the present inventiveconcept. The sensing unit may be disposed at a center of the displayarea DA or at a position on the display center line DCL and/or thesensing center line SCL. The lens unit may be disposed at a bottom DPbof a display panel (touch panel) DP. The display panel DP may be aliquid crystal device (LCD) having a screen corresponding to the displayarea DA, a circuit element to form an image to be displayed on thescreen, and a light element to generate light toward the circuitelement. The sensing unit may be disposed at a center of the LCD. Inthis case, the LCD may provide a transparent condition to a portioncorresponding to the sensing unit such that the sensing unit senses theobject through the LCD. The light element may have a portioncorresponding to the sensing unit to prevent interference with thesensing operation of the sensing unit. The display panel DP may beformed with an LED unit having a number of LEDs. In this case, the LEDunit may have a portion without an LED to prevent interference with theoperation of the sensing unit.

FIG. 17 is a view illustrating a virtual area of a sensing unit and adisplay area of a display unit according to an embodiment of the presentinventive concept. When the sensing unit is disposed at the center ofthe display area DA, the virtual area VA is disposed in the display areaDA, and the sensing center line SCL and the display center area SCL areidentical to each other.

FIGS. 18A and 18B are views illustrating a virtual point displayed on adisplay area of a display unit in an electronic apparatus according toan embodiment of the present inventive concept. When an object Ob issensed to correspond to one of icons Ma, Mb, and Mc, a symbol SYM isdisplayed at the corresponding icon Ma, Mb, or MC on the display areaDA, as illustrated in FIG. 18A. The symbol SYM representing the sensedobject may be disposed at a center portion or a side portion of an areaof the corresponding icon Ma, Mb, and Mc. When the object is sensed tobe disposed at a portion which does not correspond to one of the iconsMa, Mb, and Mc, it is possible that a symbol SYM may be displayed at apoint outside the icons MA, Mb, or Mc, as illustrated in FIG. 18B. Thesymbol SYM may be a dot, a flash, a brink, or a sign. However, thepresent general inventive concept is not limited thereto. The symbol maybe a characteristic change of the corresponding one of the icons Ma, Mb,and MC while the other icons do not show any characteristic change.

FIGS. 19 and 20 are flowcharts illustrating a method of inputting a usercommand using a sensing unit in an electronic apparatus according to anembodiment of the present inventive concept.

Referring to FIG. 19, when a sensing unit, for example, a camera, isturned on, activated, or starts one of functions thereof in operation1910, an object exists in the photographed image in operation 1920. Avirtual area is extracted from the photographed image in operation 1930.An object is extracted from the virtual area in operation 1940. Thestatus of the object is analyzed and determined as a user input when theobject is disposed at a position corresponding to an icon displayed on adisplay area in operation 1950. A function corresponding to the icon isperformed in operation 1960.

Referring to FIG. 20, when a sensing unit, for example, a camera, isturned on, activated, or starts one of functions thereof in operation2010, an object is detected from a virtual area corresponding to adisplay area in operation 2020. A coordinate of the virtual areacorresponds to a coordinate of the display area. When a position of theobject with respect to the coordinate of the virtual area corresponds toa position of an icon with respect to the coordinate of the displayarea, it is determined that the presence, existence, or movement of theobject is a user input in operation 2030. A function of the icon isperformed in operation 2040.

FIG. 21 is a view illustrating a location of a sensing unit in anelectronic apparatus 2100 according to an embodiment of the presentinventive concept. The sensing unit may be one or more sensing elementsdisposed positions 2103 a, 2103 b, 2013 c, and 2103 d around a displayarea DA of a display panel 2102 on a housing 2101. When one or morecameras are included in the sensing unit, the cameras may be installedat positions 2104 and at least one of the positions 2103 a, 2103 b, 2013c, and 2103 d. It is possible that the sensing element and the cameramay be same to photograph an object as a photographing mode and to sensean object as a user input with respect to the display area DA in asensing mode. Accordingly, the camera can be usable in one of thephotographing mode and the sensing mode or also usable in both thephotographing mode and the sensing mode at the same time. In this case,an image corresponding to the photographed object has a centercorresponding to the sensing center line SCL, and an image correspondingto the sensed object in the virtual area VA may have a centercorresponding to the display center line SCL. When the object disposedat a position on the display center line DCL is photographed anddisplayed at a position different from the display center line DCL.However, when the object disposed at a position on the display centerline DCL is sensed and analyzed to be disposed at a position on thedisplay center line SCL according to coordinates of the display area DAand the virtual area VA.

FIGS. 22 and 23 are views illustrating an electronic apparatus having asensing unit installed thereon according to an embodiment of the presentgeneral inventive concept. FIG. 22 illustrates that a sensing unit isinstalled at a side of the display area DA such that a sensing centerline SCL is perpendicular to the display area DA and a boundary line BLis inclined to spaced apart from the display area according to adistance from the sensing center line SCL or the sensing unit. FIG. 23illustrates that a sensing unit is installed at a side of the displayarea DA such that a sensing center line SCL is not perpendicular to thedisplay area DA. The sensing unit may be installed such that a boundaryline BL can be substantially parallel to the display area DA. However,the present general inventive concept is not limited thereto. Theboundary line BL may be spaced apart from the display area DA by adistance so that the sending unit can sense a user finger or tooldisposed adjacent to the display area DA. The sensing unit may have asub-housing with a lens unit having the sensing center line SC, and thesub-housing may be installed in the sub-housing at an inclination anglewith a plane of the display area DA. A method of installing a sensingunit on a housing is well known, detail description thereof is omitted.It is possible that the boundary line BL of FIG. 23 may be inclined withrespect to the display area DA. In this case, an inclination angle IAbof the boundary line BL of FIG. 23 is smaller than an inclination angleIAa of the boundary line BL of FIG. 22. A hatched portion is a virtualarea of a sensing area disposed above the display area DA in a directionperpendicular to the display area DA.

The present general inventive concept can also be embodied ascomputer-readable codes on a computer-readable medium. Thecomputer-readable medium can include a computer-readable recordingmedium and a computer-readable transmission medium. Thecomputer-readable recording medium is any data storage device that canstore data as a program which can be thereafter read by a computersystem. Examples of the computer-readable recording medium include asemiconductor device, a read-only memory (ROM), a random-access memory(RAM), a flash memory, a semiconductor chip package, CD-ROMs, magnetictapes, floppy disks, and optical data storage devices. Thecomputer-readable recording medium can also be distributed over networkcoupled computer systems so that the computer-readable code is storedand executed in a distributed fashion. The computer-readabletransmission medium can transmit carrier waves or signals (e.g., wiredor wireless data transmission through the Internet). Also, functionalprograms, codes, and code segments to accomplish the present generalinventive concept can be easily construed by programmers skilled in theart to which the present general inventive concept pertains.

As described above and also illustrated in the drawings, a user mayinput a user command on a virtual area corresponding to a display areawithout touching the display area in an electronic apparatus accordingto an embodiment of the present inventive concept. Also, the virtualarea may be spaced apart from the display area, disposed parallel to thedisplay area, and disposed to overlap the display area in a verticaldirection from the display area in the electronic apparatus such that anobject can be sensed to correspond to the same point in the virtual areaand the display area in the vertical direction according to anembodiment of the present general inventive concept.

Moreover, as described above and also illustrated in the drawings, theelectronic apparatus may include a housing, a display unit installed inthe housing and having a display area to display an image or a menuicon, a sensing unit installed in the housing adjacent to the displayunit, and having a virtual area to sense an object which is disposedover the display area of the display unit, the virtual area defined by aboundary extended from the display area in a direction perpendicular tothe display area, and a controller to analyze status of the objectaccording to an original state of the object and a sensed state of theobject, and to determine the analyzed status of the object as a userinput.

The original state of the object and the sensed state of the object withrespect to the display area may be different from each other accordingto a difference between center lines of the sensing unit and the displayarea of the display unit.

The controller may adjust the sensed state of the object to the originalstate of the object with respect to the display area to determinewhether the status of the object corresponds to the user input to thedisplay area.

The controller may control the display unit to display a symbolrepresenting the status of the object with respect to the display area.

The symbol may be displayed on the menu icon or outside the menu icon inthe display area according to a location of the status of the objectwith respect to the display area.

The virtual area and the display area may overlap in the boundary in theperpendicular direction.

The menu icon may be disposed in the display area, and the object may bedisposed in the virtual area to correspond to the menu icon of thedisplay area.

The sensing unit may have a sensing area with a sensing center line andthe display area has a display center line which is spaced apart fromthe sensing center line by a distance.

The virtual area may be disposed in a side of the sensing area withrespect to the sensing center line, and the controller may extract thevirtual area from the sensing area.

When the object moves a first distance in a first direction within thevirtual area, the sensing unit may sense the object moving a seconddistance in a second distance since a sensing center line of the sensingunit is not a center line of the virtual area, and the controller mayanalyze the status of the object according to differences between thefirst and second distances and between the first and second distancesand determines the status of the object as the user input with respectto the display area.

When the object moves toward a first position of the display area, thesensing unit may sense the object moving toward a second position of thedisplay area in the virtual area, the second position being differentfrom the first position. The controller may adjust the second positionto the first position with respect to the display area.

The sensing unit may sense the object at a first position with respectto the virtual area, and the controller may convert the sensed firstposition into a second position with respect to the display area as thestatus of the object and determines the status of the object as the userinput.

The display area and the virtual area may have a same coordinate, thesensing unit may sense the object in the coordinate of the virtual area,and the controller may determine the status of the object as the userinput from the sensed object according to the coordinate of the displayarea.

The virtual area is disposed to overlap the display area in a verticaldirection from the display area, and the controller may determinewhether the status of the sensed object of the virtual area correspondsto the menu icon of the display area, and also determine the status ofthe sensed object as the user input of the menu icon to perform afunction of the menu icon.

The sensing unit may sense one or more sub-objects, and the controllermay determine one of the one or more sub-objects as the object.

The sensing unit may sense a user face and a user finger, and thecontroller may determine the user finger as the object.

The sensing unit may sense a user finger as the object, and thecontroller determines a distal end of the user finger as the status ofthe object.

The sensing unit may have a sensing center line to have an angle with adisplay center line of the display area, and the sensing center line ofthe camera may not be parallel to the display center line of the displayarea.

The sensing unit may be a camera having a photographing area tophotograph the object. The controller may extract from the photographingarea the virtual area including the object disposed over the displayarea, and may also determine from the photographed image that the objectis disposed over the menu icon in a direction perpendicular directionfrom the display area.

The display unit may be a touch panel usable as an output unit todisplay the image or the menu icon and an input unit to receive the userinput to control the image or the menu icon. The sensing unit mayinclude a lens unit with a sensing area in which the virtual area and anon-virtual area are included, and the lens unit includes a sensor tophotograph the virtual area.

The display unit may include a display panel having the display area,and the sensing unit is disposed at a bottom area of the display panelof the display unit.

Furthermore, as described above and also illustrated in the drawings,The electronic apparatus may include a housing, a display unit installedin the housing, and having a display area usable as an output unit todisplay an image or a menu icon and an input unit to receive a userinput to control a function of the electronic apparatus including theimage or the menu icon, a sensing unit installed in the housing adjacentto the display unit, and having a camera with a photographing areaincluding a virtual area to sense an object which is disposed over thedisplay area of the display unit, the virtual area defined by a boundaryextended from the display area in a direction perpendicular to thedisplay area, and a controller to analyze status of the object accordingto an original state of the object and a sensed state of the object, andto determine the analyzed status of the object as the user input.

Furthermore, as described above and also illustrated in the drawings,the electronic apparatus may perform a method including providing ahousing with a display unit having a display area to display an image ora menu icon, and a sensing unit installed adjacent to the display unitand having a virtual area to sense an object which is disposed over thedisplay area of the display unit, the virtual area defined by a boundaryextended from the display area in a direction perpendicular to thedisplay area, and analyzing status of the object according to anoriginal state of the object and a sensed state of the object, anddetermining the analyzed status of the object as a user input.

Furthermore, as described above and also illustrated in the drawings,the present general inventive concept may be implemented as acomputer-readable medium to contain computer-readable codes as a programto perform the method described above in the electronic apparatus.

Although a few exemplary embodiments of the present general inventiveconcept have been shown and described, it will be appreciated by thoseskilled in the art that changes may be made in these exemplaryembodiments without departing from the principles and spirit of thegeneral inventive concept, the scope of which is defined in the appendedclaims and their equivalents.

What is claimed is:
 1. A mobile device, comprising: a housing having afirst side and a second side; an input and output unit comprising apanel having a display area on the first side of the housing, whereinthe display area of the panel is configured to display an image or menuto a user and to receive a user input from the user to perform afunction of the mobile device; a sensing unit disposed between the paneland the second side of the housing to sense a first object, which isdisposed to face the display area of the panel, through the panel; and acontroller disposed in the housing and configured to perform a sensingmode which includes a sensing operation of the sensing unit to sense thefirst object through the panel and to generate a sensed image from thesensed first object, and to determine a state of the sensed image asanother user input to perform another function of the mobile device. 2.The mobile device of claim 1, further comprising: a photographing unitdisposed on the housing and comprising one or more cameras to photographa second object, wherein the controller is configured to perform aphotographing mode which includes a photographing operation of thephotographing unit to photograph the second object and to control thepanel to display the photographing second object on the display area. 3.The mobile device of claim 1, further comprising: a photographing unitdisposed on the housing and comprising one or more cameras to photographa second object in a photographing mode, wherein the panel comprises atouch panel having the display area to display the photographing secondobject and to receive a touch input as the user input, the sensing unitcomprises a sensing element disposed on a side of the panel opposite tothe display area to sense the first object through the panel and togenerate the sensed image, and the sensed image is not displayed on thedisplay area of the panel.
 4. The mobile device of claim 1, wherein thecontroller controls the panel to display a symbol corresponding to thesensed image on the display area.
 5. The mobile device of claim 1,wherein: the panel displays another menu; the sensing unit senses thefirst object, which is disposed to correspond to the another menu,through the panel to generate the sensed image; and the controllerperforms the another function corresponding to the another menuaccording to the state of the sensed image.
 6. The mobile device ofclaim 1, wherein: the panel displays another menu on the display area;the sensing unit comprises a sensing element disposed to correspond tothe another menu and the first object; and the controller performs theanother function corresponding to the another menu according to thestate of the sensed image.
 7. The mobile device of claim 1, where thesensing unit comprises a sensing element disposed at a bottom of thepanel opposite to the display area to sense the first object through thepanel.
 8. The mobile device of claim 1, wherein the sensing unit isdisposed along a first center line which is parallel to a second centerline of the display area of the panel.
 9. The mobile device of claim 1,wherein: the panel comprises a liquid crystal device including a lightelement and a circuit element to form the image or menu on the displayarea; and the sensing unit comprises a sensing element disposed betweenthe circuit element of the liquid crystal device and the second side ofthe housing.
 10. The mobile device of claim 1, wherein: the panelcomprises a light emitting device unit configured to provide a firstportion with a first number of light emitting elements and a secondportion with a second number of light emitting elements, to form theimage or menu; and the sensing unit comprises a sensing element disposedat a location corresponding to the first portion of the panel.
 11. Themobile device of claim 1, wherein: the sensing unit comprises a sensingelement; and the panel comprises a portion corresponding to a locationof the sensing element of the sensing unit; and the controller controlsthe panel to provide a condition to the portion when the sensing elementof the sensing unit senses the first object through the panel in thesensing mode.
 12. The mobile device of claim 1, wherein the controlleris configured to control the display area of the panel to have acondition to prevent interference between the first object and thesensing unit in the sensing mode such that another user input isreceived and determined from the sensed image of the first objectthrough the sensing unit.
 13. The mobile device of claim 1, wherein thepanel is configured to provide a condition to prevent an interferencewith the sensing operation of the sensing unit when the sensing unitsenses the first object through the panel in the sensing mode.
 14. Themobile device of claim 1, wherein: the sensing unit comprises a sensorto sense the first object through the panel; the panel comprises a firstportion and a second portion, wherein the sensor of the sensing unit isdisposed to correspond to the second portion of the panel; and the panelprovides a transparency condition to the second portion.
 15. The mobiledevice of claim 1, wherein: the first object comprises a plurality ofsub-objects, one of which has a higher priority than the other one; andthe controller is configured to generate another sensed imagecorresponding to the one sub-object with the higher priority, and isconfigured to determine at least one of the state of the sensed imageand a state of the another sensed image of the one sub-object with thehigher priority as the another user input to perform the anotherfunction of the mobile device.
 16. The mobile device of claim 1,wherein: the first object comprises a plurality of sub-objects, one ofwhich has a higher priority than the other one; the sensing unit sensesthe plurality of sub-objects through the panel and generates anothersensed images corresponding to the other sub-object and the onesub-object with the higher priority; and the controller is configured todetermine at least one of the state of the sensed image and states ofthe another sensed images as the another user input to perform theanother function.
 17. The mobile device of claim 1, further comprising:an interface disposed in the housing and configured to communicate withan external device to transmit or receive data, wherein the controlleris configured to process the data according to the state of the sensedimage and to control the interface to transmit the processed data to theexternal device.
 18. The mobile device of claim 1, wherein thecontroller is configured to extract a portion of the sensed image anddetermine a state of the extracted portion as the another user input.19. The mobile device of claim 1, wherein the controller is configuredto modify the sensed image to generate a state of the modified image asthe another user input.
 20. The mobile device of claim 1, wherein: thesensing unit comprises one or more sensing elements disposed atcorresponding sensing areas; at least one of the one or more sensingelements senses the first object through the panel; and the controllergenerates the sensed image and determines the sensed image as theanother user input.